FPGA-based multi-view stereo system with flexible measurement setup

In recent years, stereoscopic image processing algorithms have gained importance for a variety of applications. To capture larger measurement volumes, multiple stereo systems are combined into a multi-view stereo (MVS) system. To reduce the amount of data and the data rate, calculation steps close to the sensors are outsourced to Field Programmable Gate Arrays (FPGAs) as upstream computing units. The calculation steps include lens distortion correction, rectification and stereo matching. In this paper a FPGA-based MVS system with flexible camera arrangement and partly overlapping field of view is presented. The system consists of four FPGA-based passive stereoscopic systems (Xilinx Zynq-7000 7020 SoC, EV76C570 CMOS sensor) and a downstream processing unit (Zynq Ultrascale ZU9EG SoC). This synchronizes the sensor near processing modules and receives the disparity maps with corresponding left camera image via HDMI. The subsequent computing unit calculates a coherent 3D point cloud. Our developed FPGA-based 3D measurement system captures a large measurement volume at 24 fps by combining a multiple view with eight cameras (using Semi-Global Matching for an image size of 640 px × 460 px, up to 256 px disparity range and with aggregated costs over 4 directions). The capabilities and limitation of the system are shown by an application example with optical non-cooperative surface

Suitability study for real-time depth map generation using stereo matchers in OpenCV and Python

Stereo imaging provides an easy and cost-effective method to measure 3D surfaces, especially due to the availability of extensive free program libraries like OpenCV. An extension of the application to the field of forestry was aimed at here in the context of a project to capture the elevation profile of forest roads by means of stereo imaging. For this purpose, an analysis of the methods contained in OpenCV for the successful generation of depth maps was carried out. The program sections comprised the reading of the image stream, the image correction on the basis of calibrations carried out in advance as well as the generation of the disparity maps by the stereo matchers. These are then converted back into depth maps and stored in suitable memory formats. A data set of the image size 1280x864 pixels consisting of 30 stereo image pairs was used. The aim was to design an evaluation program which allows the processing of the described steps within one second for 30 image pairs. With a sequential processing of all steps under the used test system and the usage of a local stereo matcher a processing time of 4.37 s was determined. Steps to reduce the processing time included parallelizing the image preparation of the two frames of the image pair. Further reduction in total processing time was achieved by processing multiple image pairs simultaneously and using storage formats without compression. A total processing time of 0.8 s could be achieved by outsourcing the stereo matching to the graphics card. However, the tested method did not achieve the desired resolutions in depth as well as in the image plane. This was made possible by using semi-global matchers, which are up to 10 times slower but significantly more accurate, and which were therefore used for further investigations of the forest path profile

FPGA implementation of a multi-view stereo approach for depth estimation and image reconstruction for plenoptic cameras

In this paper a concept for an algorithm for depth estimation and image reconstruction for a plenoptic camera is presented. The algorithm follows a multi-view stereo approach and is intended for an FPGA-based Xilinx Zynq Ultrascale+ SoC platform to allow for real-time processing in an embedded environment. The micro-lens array separates a complete image in many micro-images. The micro-images are considered as individual cameras and the processing is calculated in a multi-view stereo approach. To accomplish an adequate frame rate and a reasonable resolution efficient processing steps and fixed-point integer calculation are chosen. The conceptual algorithm will be implemented and tried out in an experimental setting in 2019

An investigation for process capability in additive manufacturing

This work presents an investigation for process capability in additive manufacturing (AM). Fused Filament Fabrication (FFF) is the additive manufacturing method, which is based for this verification. The typically layer-upon-layer building method in FFF has special effects to the geometrical quality and is important for comprehending the challenges in the additive manufacturing. But the continuously increasing of manufacturing quality in AM methods has enlarges the applying from rapid prototyping to rapid manufacturing or even to rapid tooling. The reliable applying of AM methods in these areas needs an evaluation and validation for process stability and is part of this paper. The necessary tolerance specification for purposing the process capability is common in mechanical engineering and is considered in results